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11.3.2 Stationary and Limiting Distributions 

Here we introduce stationary distributions for continuous Markov chains. As in the case of 
discrete-time Markov chains, for "nice" chains, a unique stationary distribution exists and it is 
equal to the limiting distribution. Remember that for discrete-time Markov chains, stationary 
distributions are obtained by solving tc = xP. We have a similar definition for continuous-time 
Markov chains. 

Let X{t) be a continuous-time Markov chain with transition matrix P(t) and state space 
S = {0, 1, 2, •••}. A probability distribution x on S, i.e, a vector x = [x Q , ttj, x 2 , •••], where 

Xj G [0, 1] and 


iES 

is said to be a stationary distribution for X(t) if 

x = xP{t), for all t > 0. 

The intuition here is exactly the same as in the case of discrete-time chains. If the probability 
distribution of A(0) is x, then the distribution of A(7) is also given by x, for any t> 0. 


Example 11.18 

Consider the continuous Markov chain of Example 11.17 : A chain with two states S = {0, 1} 
and 2 0 = 2 j = 2 > 0. In that example, we found that the transition matrix for any t > 0 is given 

by 


Pit) 


1 , 1-2 It 1 _ 1-2 It 

2 2 2 2 


- - I «-2 kt 1 , I _-2 It 

2 2 2 2 


Find the stationary distribution x for this chain. 
• Solution 

o For x = [xq, Tiq], we obtain 
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nP(t) = [tt 0 , 7Tj] 
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[7T 0 , TTj]. 


We also need 


7Tq "t" 7T | 1. 


Solving the above equations, we obtain 


7T 0 7ZJ 


1 

2 ' 


Similar to the case of discrete-time Markov chains, we are interested in limiting distributions 
for continuous-time Markov chains. 

Limitin g Distributions 

The probability distribution n = [tt 0 , 7r 1? k 2 , •••] is called the limiting distribution of the 
continuous-time Markov chain X(t) if 

7r. = lim P(X(t) =j\X(0) = i ) 

t—* oo 


for all i,j G S', and we have 


I 

j£S 


n j = 1 ' 


As we will see shortly, for "nice" chains, there exists a unique stationary distribution which 
will be equal to the limiting distribution. In theory, we can find the stationary (and limiting) 
distribution by solving nP{t) = n, or by finding Yww^ ^ Pit). However, in practice, finding 

P(t) itself is usually very difficult. It is easier if we think in terms of the jump (embedded) 
chain. The following intuitive argument gives us the idea of how to obtain the limiting 
distribution of a continuous Markov chain from the limiting distribution of the corresponding 
jump chain. 


Suppose that n = \n {) , n x , jt 2 , •••] is the limiting distribution of the jump chain. That is, the 
discrete-time Markov chain associated with the jump chain will spend a fraction n- of time in 
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state j in the long run. Note that, for the corresponding continuous-time Markov chain, any 

1 

time that the chain visits state j, it spends on average — time units in that state. Thus, we can 

A J 

obtain the limiting distribution of the continuous-time Markov chain by multiplying each Ttj by 

1 *k 

-r. We also need to normalize (divide by Y t~) to get a valid probability distribution. The 

A j A k 

following theorem states this result more accurately. (It is worth noting that in the discrete¬ 
time case, we worried about periodicity. However, for continuous-time Markov chains, this is 
not an issue. This is because the times could any take positive real values and will not be 
multiples of a specific period.) 

Theorem 11.3 

Let {X{t), t > 0 } be a continuous-time Markov chain with an irreducible positive recurrent 
jump chain. Suppose that the unique stationary distribution of the jump chain is given by 


Further assume that 


7t [?Tq, TZ\, ^2’ "" ]• 


v- Uk 

0 < Z J 7 <00 ' 

keS A k 


Then, 


7T - = lim P(X(t) = j |X(0) = 0 

t —> co 


L 


^kesT 


for all i,j G S. That is, n = [tt 0 , K\, ?r 2 , •■•] is the limiting distribution ofX(t). 


Example 11.19 

Consider a continuous-time Markov chain X(t) that has the jump chain shown in Figure 11.23. 
Assume the holding time parameters are given by = 2, 2 2 = 1, and 2 3 = 3. Find the limiting 

distribution for X(t). 
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1 



Figure 11.23 - The jump chain for the Markov chain of Example 11.19. 

• Solution 

o We first note that the jump chain is irreducible. In particular, the transition matrix 
of the jump chain is given by 


■0 1 O' 

0 0 1 
1 1 


The next step is to find the stationary distribution for the jump chain by solving 
nP = n. We obtain 


„ 1 

7T = -[1, 2, 2]. 

Finally, we can obtain the limiting distribution ofX(t) using 


^keS^k 


We obtain 
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Thus, we conclude that n = —[3, 12, 4] is the limiting distribution oi'X(t). 


<— previous 
next —> 
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